Estimating and Exploiting the Entropy of Sense Distributions
نویسندگان
چکیده
Word sense distributions are usually skewed. Predicting the extent of the skew can help a word sense disambiguation (WSD) system determine whether to consider evidence from the local context or apply the simple yet effective heuristic of using the first (most frequent) sense. In this paper, we propose a method to estimate the entropy of a sense distribution to boost the precision of a first sense heuristic by restricting its application to words with lower entropy. We show on two standard datasets that automatic prediction of entropy can increase the performance of an automatic first sense heuristic.
منابع مشابه
Shannon entropy in generalized order statistics from Pareto-type distributions
In this paper, we derive the exact analytical expressions for the Shannon entropy of generalized orderstatistics from Pareto-type and related distributions.
متن کاملA Note on the Bivariate Maximum Entropy Modeling
Let X=(X1 ,X2 ) be a continuous random vector. Under the assumption that the marginal distributions of X1 and X2 are given, we develop models for vector X when there is partial information about the dependence structure between X1 and X2. The models which are obtained based on well-known Principle of Maximum Entropy are called the maximum entropy (ME) mo...
متن کاملDetermination of Maximum Bayesian Entropy Probability Distribution
In this paper, we consider the determination methods of maximum entropy multivariate distributions with given prior under the constraints, that the marginal distributions or the marginals and covariance matrix are prescribed. Next, some numerical solutions are considered for the cases of unavailable closed form of solutions. Finally, these methods are illustrated via some numerical examples.
متن کاملEntropy Properties of Certain Record Statistics and Some Characterization Results
In this paper, the largest and the smallest observations are considered, at the time when a new record of either kind (upper or lower) occurs based on a sequence of independent random variables with identical continuous distributions. We prove that sequences of the residual or past entropy of the current records characterizes F in the family of continuous distributions. The exponential and the ...
متن کاملOn the Monotone Behavior of Time Dependent Entropy of Order alpha
In this paper we study some monotone behavior of the residual (past) entropy of order . We prove that, under some relation between the hazard rates (reversed hazard rates) of two distributions functions F and G, when the residual (past) entropy of order of F is decreasing (increasing) then the residual (past) entropy of G is decreasing (increasing). Using this, several conclusions regarding mo...
متن کامل